--- title: "Is it really Googlebot? Crawler IP verifier" url: https://proxy.wiki/tools/crawler-verifier/ type: Tool author: "proxy.wiki editorial" published: 2026-10-21 updated: 2026-10-21 site: proxy.wiki license: CC BY 4.0; quote freely with attribution to https://proxy.wiki/ --- # Is it really Googlebot? Crawler IP verifier > Check whether an IP from your logs is really Googlebot, Bingbot, GPTBot, ClaudeBot, PerplexityBot, Applebot or Amazonbot, with the evidence. ## Key takeaways - A crawler's user agent is a claim; its address is the evidence. - An address inside the operator's published IP list is genuine. - Reverse DNS counts only when the name resolves back to the same address. - googleusercontent.com is not accepted for Googlebot, because rentable cloud machines resolve under it. Paste an address from your logs and find out whether it really is Googlebot, Bingbot, GPTBot, ClaudeBot or another named crawler. This is an interactive tool; it runs in a browser at [https://proxy.wiki/tools/crawler-verifier/](https://proxy.wiki/tools/crawler-verifier/). What we check, crawler by crawler | Crawler | Operator | Published IP list | Prefixes in our copy | Reverse DNS | Operator docs | | --- | --- | --- | --- | --- | --- | | OAI-SearchBot | OpenAI | [searchbot.json](https://openai.com/searchbot.json) | 39 | Not documented | [developers.openai.com](https://developers.openai.com/api/docs/bots) | | ChatGPT-User | OpenAI | [chatgpt-user.json](https://openai.com/chatgpt-user.json) | 234 | Not documented | [developers.openai.com](https://developers.openai.com/api/docs/bots) | | GPTBot | OpenAI | [gptbot.json](https://openai.com/gptbot.json) | 18 | Not documented | [developers.openai.com](https://developers.openai.com/api/docs/bots) | | Claude-SearchBot | Anthropic | [bots.json](https://claude.com/crawling/bots.json) | 14 | Not documented | [support.claude.com](https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler) | | Claude-User | Anthropic | [bots.json](https://claude.com/crawling/bots.json) | 14 | Not documented | [support.claude.com](https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler) | | ClaudeBot | Anthropic | [bots.json](https://claude.com/crawling/bots.json) | 14 | Not documented | [support.claude.com](https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler) | | Perplexity-User | Perplexity | [perplexity-user.json](https://www.perplexity.com/perplexity-user.json) | 4 | Not documented | [docs.perplexity.ai](https://docs.perplexity.ai/docs/resources/perplexity-crawlers) | | PerplexityBot | Perplexity | [perplexitybot.json](https://www.perplexity.com/perplexitybot.json) | 8 | Not documented | [docs.perplexity.ai](https://docs.perplexity.ai/docs/resources/perplexity-crawlers) | | Googlebot | Google | [common-crawlers.json](https://developers.google.com/static/crawling/ipranges/common-crawlers.json) | 317 | googlebot.com, google.com | [developers.google.com](https://developers.google.com/crawling/docs/crawlers-fetchers/verify-google-requests) | | Bingbot | Microsoft | [bingbot.json](https://www.bing.com/toolbox/bingbot.json) | 28 | search.msn.com | [www.bing.com](https://www.bing.com/webmasters/help/how-to-verify-bingbot-3905dc26) | | Applebot | Apple | [applebot.json](https://search.developer.apple.com/applebot.json) | 24 | applebot.apple.com | [support.apple.com](https://support.apple.com/en-us/119829) | | Amazonbot | Amazon | [Amazonbot IP addresses page](https://developer.amazon.com/amazonbot/ip-addresses/) | 1,292 | Not documented | [developer.amazon.com](https://developer.amazon.com/amazonbot) | | DuckDuckBot | DuckDuckGo | [duckduckbot.json](https://duckduckgo.com/duckduckbot.json) | 486 | Not documented | [duckduckgo.com](https://duckduckgo.com/duckduckgo-help-pages/results/duckduckbot) | An address inside the operator's list is genuine. Where the operator also documents reverse DNS, an address outside its list is checked by DNS before we call it spoofed: the name must sit under the domain shown and resolve back to the same address. Where it documents only the list, an address outside the list is not that crawler. A user agent is a sentence the client writes about itself. Anyone can send `Googlebot/2.1`, and scanners do, because some sites wave crawlers through rules that stop everyone else. The address cannot be faked the same way: a request has to come from it for the connection to work. So the question “is this really Googlebot?” is a question about the address, and the operators publish what is needed to answer it. ## The two checks operators document **The published IP list.** Google, Microsoft, OpenAI, Anthropic, Perplexity, Apple, Amazon and DuckDuckGo each publish the address ranges their crawlers use. If the address is inside one of them, the crawler is genuine. The table under the form links every file we read. **Forward-confirmed reverse DNS.** Google, Microsoft and Apple also document a DNS check. Look up the address’s PTR record; the name must end in the operator’s domain; then look up that name and confirm it points back to the same address. The second step is what makes it proof. Whoever controls an address block can set its PTR record to any name, but only the operator controls what `googlebot.com` names resolve to. ``` $ dig -x 66.249.66.1 +short crawl-66-249-66-1.googlebot.com. $ dig crawl-66-249-66-1.googlebot.com A +short 66.249.66.1 ``` ## How the verdict is decided | Result | When we say it | | --- | --- | | Genuine | The address is in the operator’s published list, or it passes forward-confirmed reverse DNS under the operator’s documented domain. | | Spoofed | The operator’s own documented check says no. For OpenAI, Anthropic, Perplexity and DuckDuckGo, which document only a list, that means the address is outside the list. For the three that also document DNS, the address must fail the DNS check too. | | Could not verify | A list could not be fetched or a DNS lookup did not finish. Nothing is concluded from a check that did not run. | One domain is deliberately not accepted for Googlebot: `googleusercontent.com`. Google’s documentation lists it for user-triggered fetchers, and it is also where rentable Google Cloud machines get their default reverse DNS names. A scanner on a cloud VM therefore passes a naive “does it end in a Google domain” check. It does not pass this one. ## Check a list yourself Most lists are JSON files of CIDR prefixes; Amazon prints its list on a web page instead. Python’s standard library is enough to test an address against one: ``` import ipaddress, json, urllib.request url = "https://developers.google.com/static/crawling/ipranges/common-crawlers.json" data = json.load(urllib.request.urlopen(url, timeout=10)) nets = [ipaddress.ip_network(p.get("ipv4Prefix") or p.get("ipv6Prefix")) for p in data["prefixes"]] ip = ipaddress.ip_address("66.249.66.1") print(any(ip in n for n in nets if n.version == ip.version)) ``` ## What we do with the address you enter We check it and discard it. It travels in the body of the request rather than the URL, so it does not appear in our access logs, and it is not stored. The range files are fetched from the operators by our server and refreshed every 12 hours. If a refresh fails, the previous copy is used for up to 14 days; the result labels that evidence stale and includes when our copy was fetched. ## Frequently asked questions ### How do I verify Googlebot? Check the address against Google's published common-crawlers.json, or run a reverse DNS lookup and confirm the name ends in googlebot.com or google.com and resolves back to the same address. Google documents both methods. ### Can a spoofed crawler pass a reverse DNS check? Not a forward-confirmed one. Whoever controls an address block can set its PTR record to any name, but the forward lookup of that name is answered by the domain's owner, so it will not point back to the spoofer's address. ### Why is ClaudeBot only checked against a list? Anthropic publishes the IP addresses its crawlers use and documents no reverse DNS method. An address outside that list is not ClaudeBot. ### Do you log the addresses I check? No. The address is sent in the request body, checked, and discarded. ## Sources 1. [Google: Verifying Googlebot and other Google crawlers](https://developers.google.com/crawling/docs/crawlers-fetchers/verify-google-requests) 2. [Bing Webmaster Tools: How to verify Bingbot](https://www.bing.com/webmasters/help/how-to-verify-bingbot-3905dc26) 3. [OpenAI: Overview of OpenAI crawlers](https://developers.openai.com/api/docs/bots) 4. [Anthropic: Does Anthropic crawl data from the web](https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler) 5. [Perplexity crawlers](https://docs.perplexity.ai/docs/resources/perplexity-crawlers) 6. [Apple: About Applebot](https://support.apple.com/en-us/119829) 7. [Amazon: Amazonbot](https://developer.amazon.com/amazonbot) 8. [DuckDuckGo: DuckDuckBot](https://duckduckgo.com/duckduckgo-help-pages/results/duckduckbot)