Googlebot Confirmation Tool
Enter a list of IP addresses to confirm whether each IP is Googlebot based on a reverse DNS lookup.
How do you verify an IP address is Googlebot?
We follow the instructions specified by Google in this post. The steps are as follows:
- Run a reverse DNS lookup on the ip address provided.
- Verify that the domain name is a subdomain of either googlebot.com or google.com.
- Run a forward DNS lookup on the host name returned from step 1 and verify it matches the original ip address.
Why is it important to verify Googlebot?
During log file analysis, a common mistake is to filter server log events by user-agent. For example, a common Googlebot mobile user-agent is:
Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
However, when only analyzing logs by user-agent, it is easy to mistake other popular web crawlers (such as ScreamingFrog, DeepCrawl, Botify, etc.) with their user-agent set to Googlebot as actually being Googlebot. Therefore, the proper method should be to run each ip address with a Googlebot user-agent through this process and filter out any ip addresses that are not actually affiliated with Google.
There are millions of bots out there pretending to be Googlebot. If you're running an SEO test and need to verify Googlebot has crawled the pages in your test before measuring results, it's crucial that you're analyzing activity from the real Googlebot.